Designing Experiments to Test Planning Knowledge about Plan-step Order Constraints

A number of techniques have been developed to effectively extract and generalize planning knowledge based on expert demonstration. In this paper we consider a complementary approach to learning in which we may execute experiments designed to test hypothesized planning knowledge. In particular, we describe an algorithm that automatically generates experiments to test assertions about plan-step ordering, under the assumption that order constraints between two steps are independent of other step orderings. Experimenting with plan-step ordering can help identify asserted ordering constraints that are in fact not necessary, as well as uncover necessary ordering constraints not represented previously. The algorithm consists of three parts: identifying the space of step-ordering hypotheses, efficiently generating ordering tests, and planning experiments that use the tests to identify order constraints that are not currently represented. This method is implemented in the CMAX experiment design module and is part of the POIROT integrated learning system.